Template-based Pose Estimation and Tracking of 3D Hand Motion

The problem of initialising and tracking three dimensional human hand motion from monocular view is addressed in this thesis. We aim to solve the initialisation and tracking in a unified framework. To that end, tracking is formulated as pose estimation of human hand at every frame. The estimated poses at each frame are then combined into smooth trajectories. Template matching forms the basic building block of the proposed systems in this thesis. A template refers to the combined set of projected model features and its corresponding pose parameters of the hand model at a certain pose. Large number of templates are created by projecting a computer graphics model in an off-line process providing a discrete approximation of the pose space. The first approach builds a hierarchical Bayesian tracking system in a generative framework, which approximates the posterior distribution at multiple resolutions using templates. A tree-based representation of the distribution is presented, where the leaves define a partition of the state space with piecewise constant density. The advantage of this representation is that regions with low probability mass can be rapidly discarded in a hierarchical search, and the distribution can be approximated to arbitrary precision. The effectiveness of the technique is demonstrated by using it for tracking 3D articulated and global motion in front of cluttered background. The second approach proposes a discriminative framework where a sparse representation of the templates are learned. In order to address the problem of pose ambiguity, a one-to-many mapping from feature to state space is learned using a set of relevance vector machines. The image features are obtained by robust template matching, and the relevance vector machines select a sparse set of these templates. The regression method is applied to the pose estimation problem from a single input frame, and is embedded within a probabilistic tracking framework to include temporal information. Declaration I hereby declare that no part of this thesis has been submitted for any other degree or qualification. This dissertation is the result of my own work and includes nothing which is the outcome of work done in collaboration except where specifically indicated in the text. The dissertation contains approximately 32,000 words and 43 figures. Some parts of the work presented in this dissertation have been, or are due to be published in the following articles. • A. Thayananthan, R. Navaratnam, B. Stenger, P.H.S. Torr and R. Cipolla. Tracking with Relevance Vector Machines. To be published in European Conference on Computer Vision 2006. • B. Stenger, A. Thayananthan, P. H. S. Torr, and R. Cipolla. Model-Based Hand Tracking Using a Hierarchical Bayesian Filter. Submitted to IEEE Trans. Patterm Analysis and Machine Intelligence • A. Thayananthan, R. Navaratnam, P. H. S. Torr, and R. Cipolla. Likelihood Models for Template Matching Using PDF Projection Theorem. In 9th Proc. British Machine Vision Conference, London, UK, September 2004. • B. Stenger, A. Thayananthan, P. H. S. Torr, and R. Cipolla. Hand Pose Estimation Using Hierarchical Detection. In Intl. Workshop on Human-Computer Interaction, pages 105-116, Prague, Czech Republic, May 2004. • B. Stenger, A. Thayananthan, P. H. S. Torr, and R. Cipolla. Filtering Using a TreeBased Estimator. In Proc. 9th IEEE International Conference on Computer Vision, Vol. II, pages 1063-1070, Nice, France, October 2003. • A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla. Learning a Kinematic Prior for Tree-Based Filtering In Proc. British Machine Vision Conference, Vol. 2, pages 589-598, Norwich, UK, September 2003. • A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla. Shape Context and Chamfer Matching in Cluttered Scenes. In Proc. Conference on Computer Vision and Pattern Recognition, Vol. I, pages 127-133, Madison, USA, June 2003.